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Abstract 

The IBT TOEFL has become the principal example of online high stakes language testing since 2005. Most instructors who do the 
preparation for IBT TOEFL face two main realities: first, students are eager and highly motivated to take the test because of the 
prospective implications; and, second, specific studies would be necessary to see if instructors may be familiarized with the use of 
computers in teaching and/or language testing for other high stakes tests. Now, many standardized tests are going online as well such 
as lELTS, BULATS and others (Garcia Laborda, 2007). National Boards of Education are also considering this possibility for the 
national university entrance examination (PAU) (Garcia Laborda & Magal Royo, 2007; Garcia Laborda, 2006). However, in this 
case, teachers may not be willing or well prepared to face the challenge due to factors such as ICT familiarity. To date, many papers 
have explored the role of teacher training in their adaptation to the inclusion or implementation of ICT in the foreign language 
classroom (Zepp, 2005; Tan et ah, 2003; Rehbein et ah, 2003). However, it is necessary to consider the differences between the use 
of ICT for language testing (as opposed to general teaching) and the profile of senior high school foreign language teachers who may 
not be as flexible as their younger colleagues. This paper describes the trainees’ final attitudes of a six-month course held in Valencia 
(Spain). 26 teachers from private schools who teach many of the students who take the test every year participated in an in-service 
teacher training course in Valencia. Instructional methods principally included training in ICT strategy and use through face-to-face 
instruction and also autonomous learning, group work, synchronous and asynchronous communication, and computer based test 
design. The teachers’ responses to long discussions in a control session and to a questionnaire, along with the observations of a 20- 
hour course and the results of the familiarization with different tools show that for this type of training it is not only necessary to 
familiarize teachers with the testing tool but also with the process of creating online tests and with the testing process itself (Chapelle 
& Douglas, 2006; Stoynoff & Chapelle, 2005). Results from the platform use and the final questionnaire indicated that the majority 
of trainees reacted positively to the training and were eager to let the research team work with their students. However, they still felt 
they would not be able to use ICT for testing in their classes. 
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1. Introduction 

The IBT TOEFL has become the principal example of online high stakes language testing since 2005. Most instructors who do the 
preparation for IBT TOEFL face two main realities: first, students are eager and highly motivated to take the test because of the 
prospective implications; and, second, specific studies would be necessary, instructors may be usually familiarized with the use of 
computers in teaching and/or language testing. Now, many standardized tests are going online as well such as lELTS, BULATS and 
others (Garcia Laborda, 2007). National Boards of Education are also considering this possibility for the national university entrance 
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examination (PAU) (Garcia Laborda & Magal Royo, 2007; Garcia Laborda, 2006). However, in this case, teachers may not be 
willing or well prepared to face the challenge due to factors such as ICT familiarity. To date, many papers have explored the role of 
teacher training in their adaptation to the inclusion or implementation of ICT in the foreign language classroom (Zepp, 2005; Tan et 
al., 2003; Rehbein et al., 2003) However, it is necessary to consider the differences between the use of ICT for language testing (as 
opposed to general teaching) and the profile of senior high school foreign language teachers who may not be as flexible as their 
younger colleagues. This paper describes the trainees’ attitudes of a a six-month course held in Valencia (Spain). 26 teachers from 
private schools who teach many of the students who take the test every year participated in an in-service teacher training course in 
Valencia. Instructional methods principally included training in ICT strategy and use through face-to-face instruction and also 
autonomous learning, group work, synchronous and asynchronous communication, and computer based test design. The teachers’ 
responses to long discussions in a control session and to a questionnaire, along with the observations of a 20-hour course and the 
results of the familiarization with different tools show that for this type of training it is not only necessary to familiarize teachers with 
the testing tool but also with the process of creating online tests and with the testing process itself (Chapelle & Douglas, 2006; 
Stoynoff & Chapelle, 2005). Results from the platform use and the final questionnaire indicated that the majority of trainees reacted 
positively to the training and were eager to let the research team work with their students. However, they still felt they would not be 
able to use ICT for testing in their classes. 

2. English Teachers and their Perceptions of ICT 



It is self-evident that changes require adaptation and training. Training is not only important but necessary because it has the 
power to change trainees’ attitudes in any area. ICT and computers require teacher training for a number of reasons: 

a) Lack of familiarity of equipment such as certain computer programs; 

b) In some cases, attitudes towards change tend to be negative until trainees become familiarized with the new 
environment (in this case, with the new computer platform or software); 

c) Previous experience may be negative or nonexistent; 

d) Computer use may be informal or intuitive and dissociated from a cognitive rational approach; and 

e) Cognitive use may he biased by age, cognitive style, strategic knowledge, digital literacy or intended use. 

In a previous qualitative study with 100 PAU teachers in the Valencian province, it was shown that teachers do not use computers 
in their classes even occasionally. 93 teachers reported never having used computers with their students. About 15% felt uneasy 
about using computers in their classes and 55% completely unable to do so. Additionally, more than a half of the teachers recognized 
having little knowledge of testing as a process. Consequently, the teacher training program intended to deal with these issues: 

1) Motivate the teachers to perceive and evaluate the positive and negative aspects of the test change; 

2) Know the testing platform as students, raters and administrators; 

3) Observe whether teachers actually needed to get to know the evaluation process or simply the way in which the test was 
digitally delivered; 

4) Evaluate the process of how teachers accept or reject the introduction of the computerized test. 

3. Course Description 

The 30-hour course was titled “Specializing in PAU” and categorized as an accredited institutional workshop. However, in the 
first meeting it was agreed by the in-service teachers that it was necessary first to have some knowledge of evaluation and computers 
that they did not have at that time. In 2007, the course began with 32 teachers who represented about 24% of the total number of 
teachers of last high school year in the Valencia province. The average attendance was 23 during the nine three-hour sessions (the 
rest corresponded to a mini conference organized by the publishing sponsor) which devoted about 60% of the time to instruction in 
testing and the rest to developing their ICT skills. There was only one session in which there were very few teachers (it was held on 
the same day and time as a very important game of the city’s football team). 

From the first session it was clear that most teachers had email addresses (although a 27 year-old trainee did not, nor did he know 
how to sign up for an account). The trainer asked the teachers, in order to get familiarized with digital communication, to use their 
emails or even Skype for tutoring communication. In the end, communication was regarded very positively in terms of time 
management and administrative tasks thus enhancing learning and developing computer literacy. 

4. Analyzing the Teachers’ Attitudes and Conclusions 

This study intended to explore whether senior teachers would be willing to implement the IB PAU (Internet based University 
entrance examination) in their own classes. The main hypothesis was that after a 30-hour training course most teachers would show 
interest in trying the new test with their students. A second goal would be to verify if they would prefer this test method given the 
fact that their students may have some advantage when taking the test. In order to gather the necessary information, a combined 



Jesiis Garcia Laborda et al. / Procedia Social and Behavioral Sciences I (2009) 141-144 



143 



approach was used. At the end of the course the trainees had a control / evluation session followed by a personal questionnaire to 
gather information about their attitudes. The results are herby presented. 

The questionnaire began with one question about computer literacy and another about the use of English (as a foreign language). 
Since the scope of this paper is only related to the first, we will only mention that 15 teachers acknowledged that their computer 
literacy and skills were average (basic use of Internet and word processing). Three teachers mentioned that they thought they had 
good computer literacy and could do more advanced processes (one of them was in fact the computer class instructor). 

The main questionnaire comprised nine questions. Although the first was an ice-breaker (What do you think about the IB PAU 
format? How or why?), it was clear that teachers did not focus on the delivery means (Internet or computer based) but on the content 
itself. Since the content, level and types of questions were similar to the pen and paper test, they only mentioned that a computer 
delivered test would be not realistic. In their opinion, the current state of teaching in Spain does not meet the appropriate conditions 
of number of students, time devoted to computer exercises and the requirements of the syllabus. Four of the teachers thought that the 
format would be appropriate if it included audiovisuals and a listening comprehension task. As opposed to our expectations, only one 
respondent mentioned the interface layout and stated a preference for all the questions to be presented simultaneously instead of one 
by one (as is the case with PAUER or IB TOEFL). 

Although Question 3 (What do you think about the IB PAU format? How and why?) specifically addressed the format of the test, 
teachers reacted as if the question was related to the content and expected competence. Thus, since there are no oral tasks currently 
and yet there would be in the future test, they thought that it was too complicated and would take a few years to get used to this skill. 
Some of them mentioned the need for additional learning materials, and that for longer speeches students should be allowed to take 
notes (as they do in TOEFL). Some also mentioned that even if the change was not difficult, they may need support teachers for the 
training of test taking with computers. 

In relation to the interface design for the oral tasks. Question 4, teachers said that students would have to spend many hours on 
computer based practice plus the natural associated changes due to the fact that they do not have these tasks now, so they may need 
training to get used to speaking. They may adapt easily but the problem would still be the format and new skills rather than the 
presentation layout. 

In reference to their preparation to teach the classes with similar platforms (“How do you think the test interface would affect the 
performance in the oral tasks (“speaking”) in class and during the test?”), there was a wide range of responses. Eight clearly stated 
that it would not affect performance in any way, five that they believed it would and two more stated that if they had the appropriate 
materials, it would. 

Regarding the effects of the introduction of audiovisuals on the test. Question 6, there were great discrepancies. Some said that 
there should be no changes in objective tasks (the researcher wondered if they meant “selection or multiple choice tasks”). Seven 
thought that it would really improve the results and would also diminish routine in the classroom, but they also mentioned that the 
test should change. Four others mentioned that the audiovisuals would modify the way of assessing listening comprehension and 
speaking but, since these two skills are not currently evaluated, the change would imply major changes in the classes. Two more 
agreed that audiovisuals would benefit the students’ overall performance, two that using digital computers would benefit their 
teaching if alternated with face-to-face instruction, one that if used continuously it would distract their pupils’ attention and, finally, 
two that it would be motivating for their students. 

As for new skills to be learned, almost all mentioned strategic competence. A few also mentioned general speaking and listening 
as well as expression competences. Two mentioned note taking and only one said that the use of the digital platforms, if easy to 
handle, like this one, would not change much from today’s results. 

In connection to motivation, 16 said that the computer based test would motivate their students due to its novelty and the feeling 
of play. One said that in principle it would motivate them but that if the platform was too complex to work with, it could have the 
opposite effect, and one other questioned the platform efficacy for language evaluation. 

In the responses in relation to the effects of teacher training for the Internet based PAU (IB PAU) there was not, on the whole, a general 
position. The results indicate that even when teachers support the new format and the inclusion of new types of tasks, especially oral (mainly 
“speaking”), there is still a long way before the conditions for implementing the new delivery system can be met. There are two 
current trends: on the one hand, those who favor technology as a new way to assess skills that had never been measured before and, 
on the other hand, those who are reluctant to accept any change. Since the paper has not correlated age with results, it is difficult to 
assert that the greater age of most of the teachers included in the study makes change difficult for them and thus their motivation is 
limited. The researcher believes that these negative attitudes may be an overgeneralization and that the relation between age and 
computer skills is not conclusive in this case. Since the change in the exam requires the active addressing of both computer and face- 
to-face interactions, further studies will have to observe whether prior computer literacy is an active factor in accepting or rejecting 
the new test and/or its delivery means. It would also be relevant to observe how grammar-vocabulary classes could be transformed 
into communicative classes due to the effect of the presence of computers. 

All in all, about half of the teachers supported the change and half of them did not. Among the reasons for the disagreement 
would be the limitations of class size and class time, computer lab availability, whether students would actually accept the change. 
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and the possibility that students would not be able to adapt productively to being assessed in oral tasks. There was also a general 
agreement that teacher training seminars and courses would be necessary, and finally that the change should be legally and 
compulsorily imposed. Only one teacher mentioned that the current test would be desirable. 
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Appendix A. Questionnaire 

1. How do you consider your computer skills? ; 

2. For what and how much do you use English in your classes? 

3. Main Questionnaire 

4. What do you think about the IB PAU format? How and why? 

5. What do you think about the oral section? Do you find its delivery (format, clarity, image etc) difficult? How do you think the 
test interface would affect the performance in the oral tasks (“speaking”) in class and during the test? 

6. Do you think that at this moment [after the training sessions] you are better prepared for the teaching of languages assisted by 
digital platforms similar to this? 

7. Do you think that the introduction of audiovisuals worsens or improves the students’ performance in the PAU test? Why? 

8. What do you think your students will have to learn with this evaluation system that they do not currently learn (content or 
strategies)? 

9. Do you think this IB PAU format would motivate your students? 

10. Express your opinion for or against this change. 



